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Sir: 



BRIEF OF APPELLANTS UNDER 37 C.F.R. §1.192 (c) > 



Appellants have filed a timely Notice of Appeal from the final office action, 
mailed October 27, 2003, of the Primary Examiner in finally rejecting claims 1, 2 and 5-7 
in this application. This appeal brief is being filed in triplicate pursuant to 37 C.F.R. 
§1. 192(a). 

Please charge International Business Machine Corporation's Account No. 50- 

02/24/2004 BHflRRIse)55^q(}J^l^(gjfloufa?efl9©30.00 (37 C.F.R. §1. 17(f)) to cover the fee for filing this appeal 
01 FC:1402 330-00 Dfl 

brief Appellants believe that no extensions of time are required at this time. However, if 



additional extensions of time are necessary to prevent abandonment of this application, 
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then such extensions of time are hereby petitioned under 37 C.F.R. §1.1 36(a), and any 
fees required therefor (including any additional fees for filing of the Appeal Brief) are 
hereby authorized to be charged, or overpayment credited, to EBM Deposit Account 50- 
0510. 

REAL PARTY IN INTEREST 

The real party in interest in this appeal is Intemational Business Machine 
Corporation, assignee of the entire interest in the above-identified application. 

RELATED APPEALS AND INTERFERENCES 

The Appellants, their legal representatives and the Assignee are not currently 
aware of any appeal that may directly affect or be indirectly affected by or have some 
bearing on the Board's decision in this appeal. 

STATUS OF THE CLAIMS 

Claims 1 -7 are currently pending. 
Claims 3 and 4 are allowed. 

Claims 1, 2 and 5-7 stand rejected under 35 U.S.C. § 103(a) over U.S. Patent 
No.5,659,790 issued to Kim et al. ("Kim") in view of U.S. Patent No. 6,397,251 issued to 
Graf ("Graf) and fiirther in view of U.S. Patent No. 5,682,384 issued to Zarros 
("Zarros"). 
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The rejections of claims 1, 2 and 5-7 under § 103(a) are now the subject of the 

present appeal. The claims in issue and status of all claims are attached in Appendix "A". 



STATUS OF AMENDMENTS 

All prior amendments to the application have been entered. 



SUMMARY OF THE INVENTION 

The invention provides for a method of composing and playing multimedia 
presentations (pg 1, lines 11-13). In the invention, the method determines the factual 
durations of multimedia objects (e.g., video, audio, animation, etc) at runtime. This 
permits playback of the multimedia objects by alleviating data delivery problems such as 
those typically due to network congestion or transmission errors (pg 1, lines 11-13 and pg 
3, lines 18-19). That is, the method alleviates problems of unreliable dehvery of the 
objects in multimedia presentations (pg 3, lines 20-22). 

The invention provides a progressive time stamp resolution method that includes 
providing a player of a multimedia presentation with information that includes two labels. 
One label is for a multimedia object's start time and one label is for the multimedia stop 
end time relative to other multimedia object start and stop times (pg 7, lines 1-12). This 
information indicates how the various multimedia objects are temporally related (pg 7, 
lines 12-15) to one another. The information also includes three durations: a minimum 
duration, a maximum duration and a preferred duration (pg 7, lines 2-21, pg 10, lines 9- 
12, and Fig. 2). By using this information, the duration of the multimedia objects is 
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resolved using the information based on actual multimedia object durations and actual 
delayed arrival time of information of the multimedia objects to be played (at least pg 7, 
line 22 to pg 10, line 8). The actual delayed arrival time is an absolute difference between 
a known duration and the preferred duration (e.g., ideal duration as established by the 
author) of the multimedia objects (pg 10, lines 17-22, pgs 10-14, Figs. 3-6). 

Thus, the flexible timing information can be used by the client to adapt the timing 
of the ongoing presentation to the environment. This will allow the multimedia objects to 
have more room to remain within an author's intent and expectations of how the 
presentation should be played (pg 4, lines 1-4). Also, by using the method of the 
invention, the resolution of the actual label time can be made progressive over time as 
more information becomes available in the form of factual multimedia objects durations, 
and arrival of information objects that are to be played in the near future (pg 10, lines 17- 
22). In this manner, the method can calculate arrival times in an expedient manner and 
compensate for any delays in transmission. 



ISSUES 

Whether the subject matter of claims 1, 2 and 5-7 is obvious in view of the level 
of ordinary skill in the art under 35 U.S.C. § 103(a) as evidenced by U.S. Patent No. 
5,659,790 issued to Kim et al ("Kim") in view of U.S. Patent No. 6,397,25 1 issued to 
Graf ("Graf ) and fiirther in view of U.S. Patent No. 5,682,384 issued to Zarros 
("Zarros"). 
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GROUPING OF CLAIMS 

The rejected claims stand or fall together. 

ARGUMENT 

The invention provides many meritorious advantages over the prior art systems. 
For example, the invention provides flexible time stamping of objects. This permits 
flexible adaptation to delivery problems typically due to network congestion or 
transmission errors. This flexible timing provides duration of an object to be specified as 
a range and the start and end times are made relative to other multimedia object start and 
end times. This information may be used by a client player to adapt the timing of an 
ongoing presentation, involving the interconnected multimedia objects, to the 
environment (e.g., transmission and network issues), while still remaining within the 
author's intent and expectations for the presentation. This provides great advantages over 
known systems, which cannot adapt readily to the environment and hence compensate, 
progressively, for jitter and timing problems. 

For example, at page 7, of the specification, an illustration shows the use of labels 
and temporal relationships between multimedia objects. This provides the advantage of 
being able to resolve the duration of the multimedia objects using the information based 
on actual multimedia object durations and actual delayed arrival time of information of 
the multimedia objects to be played. This will permit the multimedia objects to act in 
accordance with an author's intent and expectations, compensate for any delays in 
transmission and provide a jitter fi-ee presentation. . 
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Specifically: 

"consider a presentation that is authored having three 
multimedia objects, a video clip (V), an audio clip (A), and 

a background image (B) [T]the Isis authoring system 

requires the author to specify for each multimedia object 
the duration range, as well as a relative start and end time. 
For the three objects in our exemplary presentation, the 
parameters are authored as: 





start 


end 


minimum 
duration 


preferred duration 


maximum 
duration 


V 


PI 


P2 


3 seconds 


4 seconds 


5 seconds 


A 


P2 


P3 


3 seconds 


4 seconds 


4 seconds 


B 


PI 


P3 


7 seconds 


7 seconds 


8 seconds 



The labels PI, P2, and P3 are to indicate how the various 
multimedia objects are [temporally] related. This means, 
for example, that objects V and B start at the same time. 
The temporal aspect of this authored presentation can be 
depicted more clearly in Figure 2." 



Now, using the labels of the invention, an example at page 10, shows the use of 
the labels and resolution of the multimedia objects as progressive over time. By using the 
labels and temporal relationships , the absolute time of a certain label will become known 
when playback has finished for a particular multimedia object. And, very advantageously, 
as more information becomes available in the form of factual multimedia object 
durations, and arrival of informafion of objects that are to be played in the (near) fiiture, 
the system can use this information in a progressive maimer to ensure proper playback 
regardless of any transmission problems. 

By way of an illustrative example. 
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"A client (i.e., a player of the multimedia 
presentation) must receive for each multimedia object five 
items of information. These items are the two labels, one 
for the object's start time and one for the end time, and the 
three durations, the minimum, maximum, and the preferred 
duration. Li the case of video, audio, and other multimedia 
objects that have a playback speed, the preferred duration 
must correspond to the "regular" playback speed of the 
object. The information on a particular multimedia object 
must be delivered to the client prior to starting playback of 
the object. 

When playback has finished for a particular 
multimedia object, the absolute time of a certain label will 
become known. This means, that one or more label times 
can be resolved using this new information. The time stamp 
resolution is therefore progressive over time, as more 
information becomes available in the form of factual 
multimedia object durations, and arrival of information of 
objects that are to be played in the (near) future. 

To resolve the actual label time, and the 
corresponding duration of the multimedia objects that have 
that label for their respective end times, the following steps 
are taken: 



1 . Collect all the dependency relations for the 
label Px, by taking all objects n that have Px as the label for 
their end time..." 



Thus, by using the information provided by the method of the invention, playback 
of the multimedia objects can be efficiently provided by alleviating data delivery 
problems such as those typically due to network congestion or transmission errors. This 
will also allow the multimedia objects to have more room to remain v^thin an author's 
intent and expectations of how the presentation should be played. Additionally, these 
examples clearly show that labels are provided and used by the invention to temporally 
correlate objects during playback and once an object playback has finished (i.e., actually 
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played) the absolute time will become known and that one or more label times can be 
resolved over time. 

Independent Claim 1 

These same features are not provided by the references, as combined by the 
Examiner, In particular, in rejecting claim 1, the Examiner asserted that the Kim 
reference shows all of the features of independent claim 1 except for resolving the 
durations of multimedia objects using the information based on actual multimedia object 
durations and actual delay arrival time of information of the multimedia objects to be 
played. However, the Examiner was of the opinion that Graf shows this feature missing. 
The Examiner admitted, though, that the combination of Kim and Graf do not explicitly 
teach resolving durations of multimedia objects using the information based on actual 
multimedia objects durations and delay arrival time of information of multimedia objects 
to be played. But, the Examiner was of the opinion that Zarros teaches these features, 
and it would have been obvious to combine Graf and Zarros with Kim to achieve the 
claimed invention. 

Appellants agree that Kim and Graf do not show the claimed features as noted by 
the Examiner. But Appellants further submit that additional features of the claimed 
invention are also not contemplated by Kim, Graf and Zarros. Appellants also disagree 
that Graf shows using the information based on actual multimedia object durations; 
instead, Graf shows the use of preferred object durations, not actual durations. Actual 
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duration includes the result from any real delays in transmission and processing before 
and during playing by the player. 

Further, Appellants disagree that Zarros teaches or suggests resolving the 
durations of multimedia objects using the information based on actual multimedia 
durations and actual delayed arrival time of information of multimedia objects to be 
played. Zarros, in contrast, shows compensating for network jitter in arrival times of 
packets based on an estimated reference time (col. 5, 53-56). An estimated arrival time is 
clearly not an actual time, as calculated and used by the claimed invention, hi fact, using 
an estimated delay time will not provide the greater accuracy as can be ahived by the 
presently claimed invention. 

Kim generally relates to composing and playing multimedia documents with 
variable play time on a computer system and, more particularly, to composing and 
playing multimedia episodes in multimedia documents so that they are presented 
correctly in time when the document play time is varied. At column 4, Kim describes 
composing a "multimedia story" by a user manually selecting among a list of episodes 
(e.g. video, text, and audio (lines 9-10)), each associated with a triplet of lengths (i.e., 
minimum, maximum, and optimum lengths). Once these durations are manually set by 
the user using a user interactive screen shown in Fig. 2, the episodes are then scheduled 
together (col. 4, line 66 to col. 5, line 1). 

However, Kim does not teach the receiving of these episodes and resolving the 
durations based on actual multimedia object durations and delayed arrival times, the main 
problem being address by the invention. Instead, Kim teaches creating objects from a list 
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and assigning a minimum, maximum and an optimum length of each object to provide 

temporal definitions relative to each object. However, there is no teaching, whatsoever, 

on receiving these objects (e.g., after a transmission) and resolving relationships between 

objects using actual multimedia object durations and delayed arrival times. 

Also, on page 3 of the Office Action, dated October 27, 2003, the Examiner cites 

col. 6, lines 3-12 and coL 6, lines 54-63, respectively, as evidence that Kim shows 

information comprising two labels , one for a multimedia object's start time and one for 

the multimedia object's end time relative to the other multimedia object start and end 

times. Appellants respectfiiUy submit that his is incorrect. Review of these sections 

clearly shows that there is no teaching whatsoever of information supplied to a player that 

includes two labels, one for start time and one for end time as recited by the invention. 

Rather, this section simply shows a flowchart/process of Fig 6 which has a starting step 

at step 601 (line 11) and an ending step (line 63), respectively. That is, Kim simply 

shows that a process "starts" for creating a multimedia story and shows that the process 

"ends". This does not show two labels; namely, 

(i) a label for a multimedia object's start time relative 
to other multimedia object start times; and 

(ii) a label for the multimedia object's end time, 
relative to other multimedia object end times. 

Further, both Graf and Zarros also fail to teach or suggest these features. 
For example, on page 4 of the October 27, 2003 Office Action, the Examiner 
admits that the combination of Kim and Graf do not explicitly teach 
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"resolving the durations of multimedia objects using 
information based on actual multimedia object durations 
and actual delay arrival time of information of multimedia 
objects to be played." 

Appellants certainly agree. 

However, the Examiner also then contradicts this statement by asserting that Graf 
does show the above feature. In this contradictory statement, the Examiner directed 
Appellants' attention to col. 4, lines 15-33 and col. 5, lines 49-56. However, Appellants 
maintain that Graf does not teach using actual multimedia obiect durations (i.e., played 
durations). Instead, Graf teaches using preferred durations (i.e., desired or ideal durations 
prior to actual playback). 

More specifically, Graf is directed to a system for compensating for transmission 
jitter and burstiness in packets when transmitting a file of data such as video over 
networks. At col. 4, lines 15-33 and col. 5, lines 49-56, Graf is describing that an 
additional time delay is being pre-calculated off-hne (i.e., pre-calculated before 
dehverv/transmission of anv data ) (col. 4, line 19-23) and the pre-calculated delay 
parameter is appended to each of the multimedia files in an attempt to compensate for 
any possible delays/jitter based on a model of the underlying network. This attempts to 
accommodate a particular network's bandwidth and possible situational delays such as, 
for example, jitter. The pre-calculation is performed using a presentation file to be served , 
i.e., before dehvery of the file (coL 4, 39-43). Since this file has not yet been sent, when 
the pre-calculation is performed, the file has not yet experienced any delays, and 
therefore cannot be representative of an actual duration. This is the exact reason that 
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Graf can only use a preferred duration (i.e., a duration resulting from being created). That 

is, the file necessarily has only a preferred duration since it has not been transmitted yet 

and encountered delays form the transmission. 

As is calculated by the present invention, a file only achieves an actual duration, 
once it has been transmitted, received, and played by a player. Only then can the actual 
duration become a known duration, as taught by the invention. Actual duration includes 
the result from any real delays in transmission and processing before and during playing 
by the player. In Graf, no actual durations are determined, since there is no contemplation 
of durations after transmission and playing of the file. Graf is based on a model and pre- 
determined calculations. 

Further, Graf does not teach or suggest determining an actual duration of a 
multimedia obiect since an actual duration is not known until a player has been supplied 
an object and played. In Graf, there is no indication at all that an actual duration is ever 
known (e.g., playback has ever occurred by a player and a duration becoming known). In 
Graf, the file is scanned by a server, not a player (col. 4, line 39-43) and the modeling 
used by Graf is an estimation process before any actual transmission of data ever occurs. 
By logic, this can never be an actual duration as taught by the invention. 

Also, Graf is concerned with only one file or object at a time in the pre- 
calculation and, in fact, never contemplates resolving durations between objects. The 
invention, on the other hand, requires multiple objects to be played in relation to one 
another. It is the use of the multiple objects that temporal and progressive time stamping 
can occur, a great advantage over the Graf model. Graf is not concemed with relational 
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timing of objects in a presentation. There is simply no suggestion or teaching in Graf that 

the actual delayed arrival time is an absolute difference between a known (e.g., played) 

duration and the preferred duration of multimedia objects (e.g., as supplied in the labels). 

Additionally, the invention is defining the actual delayed arrival time in absolute 
terms based on a played object (i.e., a known duration). Nowhere does Graf deal with 
relationships of multiple objects and does not teach or suggest an actual delayed arrival 
time as being an absolute difference between a known duration (i.e., played) and the 
preferred duration of the multimedia objects (as suppHed by labels). Additionally, the 
invention teaches at least two durations, actual and preferred, and an actual delayed 
arrival time is an absolute difference between a known duration and the preferred 
duration of the multimedia objects. Graf does not teach or suggest such a calculation. 

The Examiner further argues that Zarros explicitly teaches durations of 
multimedia objects using information based on actual multimedia object durations and 
actual delay arrival time of information to be played. Li this argument, the Examiner 
directs Appellants' attention to to col. 5, lines 19-46 of Zarros. Appellants again 
respectfiiUy disagree. Zarros simply does not teach durations of multimedia objects using 
information based on actual multimedia object durations and actual delay arrival time of 
information to be played. 

Rather Zarros teaches compensation of packet jitter over a network. But, first, 
packets are not multimedia objects, as one of ordinary skill in the art would recognize. 
Typically, packets simply bear/transport pieces of objects (among other data), but are. 
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themselves, not multimedia objects. Further, packets are typically quite unaware of the 

content that they bear such as, for example, multimedia objects. 

In any event, Zarros does not teach calculating relational durations of multimedia 
objects at all. Instead, Zarros deals with jitter of packets by using an estimation method 
(see at least, col. 2, lines 44-46, col. 5, lines 53-56, col. 6, lines 42-55, col. 7, 1-4, col. 7, 
lines 61-63). Since the compensation of jitter is based on estimations on a per packet 
basis, then there logically cannot be any actual and known duration of multimedia objects 
as taught by the invention. Zarros teaches a mechanism at a receiver to estimate playback 
time of packets using the concept of reference times or average arrival times which is 
achieved by modeling traffic of the packets as noncontinuous periodic. This means, even 
if packets are generated aperiodic, the sender will insert "empty data" in the packet to 
make the generation times of packets periodic. The receiver may compensate for jitter in 
packet reception by using the estimated reference time from a given source to 
compensate for any packet delays relative to the estimated reference time. 

Appellants also note that the Examiner is attempting to assert that calculating and 
adjustment for packet jitter based on frequency mismatches between clocks in a sender 
and a receiver using an average delay in packet arrival (col.5, lines 19-27) is the same as 
resolving the duration of multimedia objects using information based on actual 
multimedia object durations (i.e., played) and actual delayed arrival time of information 
of multimedia objects to be played, the actual delayed arrival time delay arrival time 
being an absolute difference between a known duration (i.e., played) and the preferred 
duration (i.e., ideal duration) of the multimedia objects. Nowhere are these concepts 
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present in Zarros. Further these concepts are not present in any of the references, either 

singly or in combination. 

In conclusion, the combination of references does not at least teach or even 

remotely suggest at least: 

(i) actual and known durations of multimedia objects, 

(ii) determining actual delayed arrival time as an absolute difference 
between a known duration (i.e., played) and the preferred duration 
of multimedia objects, and 

(iii) resolving temporal relations between multimedia objects based on 
(i) and (ii). 

Appellants thus submit that a prima facie case of obviousness has not been established, 
either singly or in combination, with the references of record. 
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CONCLUSION 



In summary, the combination of Kim Graf and Zarros do not teach or suggest the 
features of the claimed invention. Therefore, the references do not provide evidence that 
would support a conclusion of obviousness under 35 U.S.C. § 103(a). Appellants thus 
respectfully submit that the rejections of claims 1, 2 and 5-7 are in error and reversal 
thereof is respectfully requested. 



McGuireWoods, LLP 
1750 Tysons Boulevard 
Suite 1800 

McLean, VA 22102-4215 
Tel: 703-712-5341 
Fax: 703-712-5194 



Respectfully submitted, 




Andrew M. Cal3ermf 
Reg. No. 38,093 



Charles J. Gross 
Reg. No. 52,972 
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APPENDIX "A" 
CLAIMS 

A copy of all entered claims and a status of the claims is provided below. 

1. A method of progressive time stamp resolution 
in a multimedia presentation comprising the steps of: 

supplying a player of a muhimedia presentation with information 
comprising two labels, one for a multimedia object's start time and one for the 
multimedia object's end time relative to other multimedia object start and stop 
times, and three durations, a minimum duration, a maximum duration and a 
preferred duration for each multimedia object prior to starting playback of the 
multimedia object; and 

resolving the duration of multimedia objects using said information based 
on actual multimedia object durations and actual delayed arrival time of 
information of multimedia objects to be played, the actual delayed arrival time 
being an absolute difference between a known duration and the preferred duration 
of the multimedia objects. 

2. The method of progressive time stamp resolution in a multimedia 
presentation recited in claim 1 wherein the step of resolving comprises the steps 
of: 

calculating minimum and maximum end times for over all multimedia 
objects; 

calculating actual end times that are shared by all multimedia objects; and 
recalculating a preferred duration of each multimedia object. 

3. A method of progressive time stamp resolution in a multimedia 
presentation, comprising the steps of: 
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supplying a player of a multimedia presentation with information 
comprising two labels, one for a multimedia object's start time and one for the 
multimedia object's end time relative to other multimedia object start and stop 
times, and three durations, a maximum duration and a preferred duration for each 
multimedia object prior to playback of the multimedia object; and 

resolving the durations of the multimedia objects using said information 
based on actual multimedia object durations and arrival of information of 
multimedia objects to be played, wherein the step of resolving comprises the steps 
of: 

collecting all the dependency relations for a label Px, by taking all objects 

n that have Px as the label for their end time: 

tn + minimum(n) < tx<tn + maximum(rt) w = 1 , . . . ^ 
where U is the start time of object and A^^ is the number of 
objects; 

using the N relations to calculate the tightest bounds on tx: 
min {tx} < { tx} < max {tx} 
with 

min {tx} = max {tx + minimum(/2)} n = 1 , . . . ,// 
max {tx} = min {tx + maximum(«)} n = 1 , . . . ,A^; 
recalculating bounds on the duration of each object n, by using: 
duration(rt)== tx-tn 

to get 

min{tx} - tn < duration(n) < max{tn}- tn w=l,. . .N; and 
recalculating the preferred duration of each object n according to the 
process: 

if (preferred(rt) < niin{tx} - tn) then 
preferred(«) = min{tx} - tn 
else if (preferred (n) > max{tx}- tn) then 
preferred («) = max{tx}- tn 
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end if. 

4. The method of progressive time stamp resolution in a multimedia 
presentation recited in claim 3 wherein the step of resolving further comprises the 
steps of: 

using as the general error criterion for resolving the duration of each 
multimedia object: 

N 

E =^ {duration(w) - preferred(w)}^ 
or, substituting duration(rt) = tx-tn: 

N 

£ =^ - ^/i - preferred(n)}^ 

and taking the derivative ofE with respect to txy and setting this to 0 to obtain the 
optimal solution for the absolute time tx of label Px as: 

1 ^ 

tx= — ^ {^/j preferred(«)}; and 

calculating the corresponding duration of multimedia object n as: 
duration(rt) = tx-tn^ 

5. The method according to claim 1, further comprising the step of playing 
said each multimedia object. 

6. The method according to claim 1, wherein said actual multimedia object 
durations are larger than a preferred duration. 

7. The method according to claim 1, wherein said actual multimedia object 
durations are smaller than a preferred duration. 

\\C0M\389673.3 
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